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Key recommendations 


The assessors noted the following priorities for this initiative. We understand that some of the 
identified priorities may seem out of scope for the core delivery team and may need executive level 
support. These have been sectioned accordingly. 


For delivery team 
As you move forward into the next phase of service design: 


1. The service has not yet been tested with users with disability needs. While it's good to hear a 
plan to do so is being developed, it's imperative that the testing happens and the feedback is 
incorporated before going live. 

2. The team has done a great job setting KPIs, measuring results and making changes based on 
the data. The team has set goals of improving on baseline data by 10 %. As a next step 
consider more rigour around setting targets. 

3. While the service tests well, the assessors noted some potential usability issues (identified 
below under ‘Ensure users succeed the first time,’ that the team can address as they move 
ahead. 


For executive leadership 


1. With ownership transferring after launch there is always some risk that the hard work of the 
delivery team will not be adequately maintained and extended. The team has a good 
sustainment plan, but it’s important that ServiceOntario continue to further develop its 
digital capacity as is outlined in the submission. 


What the team needs to explore going forward 


e The team should continue to test the service with users and make improvements based on 
feedback. 

e The team should continue to leverage learnings and co-ordinate with similar Service Ontario 
services. 


What the team needs to explore going forward 


e With ownership transfer after launch there is always some risk that the hard work of the 
delivery team will not be adequately maintained. The team has a good sustainment plan, but 
it's important that ServiceOntario further develop its digital capacity as is outlined in the 
submission. 


What the team needs to explore going forward 


e The team should continue to review information with stakeholders - contact centres, MTO, 
and application teams to ensure consistency across all channels. 


What the team needs to explore going forward 


e Continue to conduct user research as the service evolves in order to understand the user 
journey. 
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What the team needs to explore going forward 


e In our walk through of the service the assessors noticed some possible usability concerns 
that the team can investigate: 

o There should be a clear statement that this is an “automated service and you're talking 
to a virtual assistant”. 

o After hearing the customer's issue, a best practice is to repeat their question / 
rephrase the question to ensure that we are understanding them properly. 

o If yes, mentioning our “willingness” to help. Then providing them the answer. If not, try 
to ask for clarification. On the second (or third) failure, porting them to a real agent is 
a good best practice. 


What the team needs to explore going forward 


e The team seems to have good agile procedures in place. Again what is important is 
maintaining and growing these ways of working as things transition to sustainment. 


What the team needs to explore going forward 


e As the team mentioned, this isn’t a‘one and done’ scenario but rather requires continuous 
testing. 


What the team needs to explore going forward 


e It's imperative that the team test the service with a diverse set of users with disability needs. 

e The team should look at their existing data and call logs to identify any trends or insights 
related to difficulties experienced by people with disabilities (can be inferred from 
speech/missed utterances, deferring to TTY, deferring to agent). 

e Consider doing a limited pilot asking callers to disclose if they use any accessibility 
accommodations in their daily life and reporting on call experience for individuals that use 
accommodations. 

e Reach out to the ACOE team for further advice. 


What the team needs to explore going forward 


e As mentioned earlier, the team should make it more clear to users how to transfer to 
a live agent. 


What the team needs to explore going forward 


e The team has set goals of improving on baseline measurements by 10 %. While it’s great to 
see success criteria, the team might want to apply more rigour in setting targets. 

e Use the Digital Performance Measurement resources to help identify KPIs for the service and 
overall outcomes. 


